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ABSTRACT 



A system and method for providing on demand access to 
information related to a movie while the movie is being 
presented to a user, where the movie was generated from the 
movie related information, are described. The system oper- 
ates by presenting the movie to the user, and then receiving 
from the user a query pertaining to the movie. The system 
determines a frame of the movie that was being presented to 
the user when the user issued the query (the system may 
extract this information from the query, or may extract this 
information from the movie itself). The system identifies, as 
specified by the query, portions of the movie related infor- 
mation relating to the frame, and retrieves those portions of 
the movie related information. These retrieved portions of 
the movie related information are presented to the user. Also 
discussed is a manager to enable users to create personalized 
versions of movies, and personalized collections of items. 
Also discussed is a system and method for providing to users 
on demand access to merchandise information related to a 
movie, and for providing on demand connectivity to 
merchants, while the movie is being presented to the users. 

29 Claims, 20 Drawing Sheets- 
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SYSTEM, METHOD, AND COMPUTER These above-described commercial opportunities associ- 

PROGRAM PRODUCT FOR PRESENTING atec * with the dissemination of information, and the prob- 

MULTIMEDIA OBJECTS, INCLUDING lcms tnat prevent such commercial opportunities from being 

MOVIES AND PERSONALIZED realized, shall now be described more concretely by con- 

COLLECTIONS OF ITEMS 5 sidering a real-life example: the motion picture industry. 

Viewing a movie via a theatrical presentation today 

CROSS-REFERENCE TO OTHER involves travelling to the theater, waiting in line to buy a 

APPLICATIONS ticket, choosing a seat, and waiting for the movie to begin. 

The viewer's schedule must match that of the theater. Once 

The present application is a continuation of Ser. No. the movie has started, the viewer must watch promotional 

08/840,276 filed Apr. 14, 1997, which is now U.S. Pat. No. previews of other films before watching the feature film. 

5,781,730, which is a continuation of Ser, No. 08/407,305 During the presentation, the viewer has no control over the 

filed Mar. 20, 1995, abandoned, herein incorporated by sound volume or presentation flow. When the movie is over, 

reference in their entireties. the viewer travels back home. The picture quality and sound 

The following applications of common assignee contain ^ are excellent and the viewer enjoys the best viewing expe- 

some common disclosure, and are believed to have an nenw possible today. However, mese benefits are ob tamed 

effective filing date identical with that of the present appli- ^ at ex P e f se of convenience and viewing control, 

cation Thus, commercial opportunities exist in electronically trans- 
ferring movies to viewer's homes, 

"System and Method for Linlang and Presenting Movies ^ technol to electronically transfer movies to view- 

With Their Underlymg Source Information, incorporated 20 er > s homes exists today viewing a movie over a ^v^. 

herein by reference. tional network or premium channel broadcast is simpler and 

"System and Method for Providing Merchant Information more convenient, but the viewer has fewer movie choices 

and Establishing Links to Merchants While Presenting a and enjoys a more limited quality experience. Like a the at - 

Movie," incorporated herein by reference. rical viewing, the viewer's schedule must match that of the 

25 broadcast and the viewer does not control the presentation 

DESCRIPTION flow. There is no travelling involved, and the cost is gener- 

1 Technical Field a ^ * ess t ^ ian mat °^ a * neater experience. The video and 

■ . audio quality are dependent on the viewer's television and 

The present invention relates generally to data linking and SQUnd systcm Note ^ ^ this vicwing optkm (and also 

presentation, and more particularly to linking and presenting ^ q ^ me theatrical presentation option discussed above), the 

movies with their underlying source information. viewer is typically not provided with any supplemental 

2. Related Art information about the movie, such as information about the 

The Information Age that currently exists has produced making of the movie, interviews with the director and cast 

incredible amounts of data that are currently unavailable to members, information on the actors, information about 

the public. In many cases, the producers of such information 35 s P ec ial effects, etc. 

would be more than willing to provide the information to the Alternatively, a person can watch a movie by renting a 

public for some reasonable fee, and the public would be vidco ta P e or lascr disk. Watching a movie in this manner 

willing to pay this fee. Typically, however, these information involves two trips to the rental shop, a search of their 

producers (also called information providers) are not able to available titles, a wait m line and a rental fee. The viewing 

take advantage of this significant business opportunity due „ experience is improved over that of a broadcast presentation 

#*u ui-u «• j- *u • f ** I. u as the viewer has some presentation controls (pause, fast 

to the problems inherent in providing the lntormation to the - , • j * \ a i j-i i £ 1 

. forward, rewind, etc.). A laser disk may also oner supple- 

P * .... mental information about the film. Parallel audio tracks may 

One problem relates to the medium in which the infor- fc e mc i u ded containing, for example, director commentary, 

mation is stored. Today, information is stored on paper, on Separate chapters of video may also be provided. However, 

film, on video, on compact disks, on magnetic tapes, on 45 there is no ability to play video content from different 

computer floppy disks, etc. Information related to a partial- chapters simultaneously. The cost is generally higher than 

lar topic may be stored on any combination of these storage that of a network broadcast. Cost comparisons to that of a 

mediums. In order to provide information on that topic to an premium channel are dependent on usage patterns (flat fee 

user, it would be necessary for an information provider to versus fee per rental). 

identify and locate all items containing information related 50 A mid-90s 'near video on demand' viewing experience 
to the topic, and then transport these items to the user. The represents something between that of broadcast movies and 
value of these items to the user is questionable, since the user a VCR/laser disk. A viewer selects a movie from an 
probably does not possess the equipment needed to access expanded broadcast rotation (generally a much smaller 
the information stored in all of the items. Accordingly, it is collection than that available at a VCR rental outlet). The 
difficult to provide information to users because such infor- 55 viewer must wait a few minutes until the next rotation of the 
mation is stored in many different storage mediums. movie broadcast. A purchase request via remote control or 
A more significant problem results from the sheer volume an 800 number is required to enable viewing. Once the 
of information relating to any particular topic. Such infor- presentation has begun, the viewer has a limited range of 
mation is of little value to an user unless the user is able to controls over the presentation flow. If the movie is paused, 
efficiently and expeditiously make use of the information. 60 the viewer must wait until the next instance of the movie at 
For example, suppose that the user is reviewing a document that point is available in the rotation. This latency is depen- 
written by an author and decides that it would be interesting dent on the number of channels available and the collection 
to study other works the author has had a part in creating. size configured by the viewer's cable carrier, and is typically 
The information provided to the user from the information about five minutes. No travel is required, and the per use fee 
provider will be of little value unless the user is able to 65 is typically added to the viewer's monthly cable bill. Supple- 
efficiently, effectively, and expeditiously traverse the infor- mental information about the movie is typically not avail- 
mation and locate the data on the author of interest. able. 
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Amid-90s 'true video on demand' viewing experience is 
effectively like that of a VCR rental, without two trips to the 
rental outlet. A digital movie server and ITV infrastructure 
are used to enable the viewer to select a movie from a large 
collection of all digitized films available on the server. The 
available selection list is not restricted by a broadcast 
rotation, and each viewer has full and immediate control 
over their movie presentation. Like 'near video on demand,' 
a purchase request via remote control or an 800 number is 
required to enable viewing. The per use fee is typically 
added to the viewer's monthly bill, and supplemental infor- 
mation is typically not available. 

Thus, the current options for viewing a movie are limited 
because they are inconvenient (the viewer must travel to the 
theater or to the video store), and/or provide the viewer with 
little or no presentation flow. Also, these current viewing 
options are limited because they provide the viewer with 
little or no supplemental movie information. Thus, the 
motion picture industry is not taking advantage of the 
potential commercial opportunities associated with provid- 
ing supplemental movie information to viewers. Even in 
those cases where supplemental information is provided, the 
viewer has little control over how such supplemental infor- 
mation is presented. Accordingly, the supplemental infor- 
mation is available only in a limited way to videophiles who 
are willing to pay extra for "special" or "director's cut" 
editions of the titles. 

DISCLOSURE OF INVENTION 

The present invention is directed to a system and method 
for providing on demand access to information related to a 
movie while the movie is being presented to a user, where 
the movie was generated from the movie related informa- 
tion. The invention operates by presenting the movie to the 
user, and then receiving from the user a query pertaining to 
the movie. The invention determines a frame of the movie 
that was being presented to the user when the user issued the 
query (the invention may extract this information from the 
query, or may extract this information from the movie itself). 
The invention identifies, as specified by the query, portions 
of the movie related information relating to the frame, and 
retrieves those portions of the movie related information. 
These retrieved portions of the movie related information 
are presented to the user. 

Hie present invention is also directed to a system and 
method for enabling a user to create, modify, and utilize a 
personalized version of a movie. The system includes a 
foundation information database which stores movie related 
information used to generate the movie. This embodiment of 
the invention operates by enabling the user to modify a copy 
of an edit decision list associated with the movie. The edit 
decision list includes an entry for each scene, clip, and take 
in the movie. The modified edit decision list defines the 
personalized version of the movie. The personalized version 
of the movie is presented to the user in response to a request 
to do so from the user. Such presentation operates by 
retrieving an entry from the modified edit decision list, 
retrieving from the foundation information database one or 
more digitized frames specified by the retrieved entry, and 
presenting the digitized frames to the user. 

In a similar manner, the present invention is directed to a 
system and method for enabling a user to create, modify, and 
utilize a personalized collection of items. This embodiment 
of the invention operates by enabling the user to modify a 
personalized collection table, where the personalized col- 
Lection table defines the personalized collection and has an 
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entry for each item in the personalized collection. The 
personalized collection of items is presented to the user in 
response to a request to do so from the user. Such presen- 
tation operates by retrieving an entry from the personalized 

5 collection table, retrieving from the foundation information 
database foundation information specified by the retrieved 
entry, and presenting the retrieved foundation information to 
the user. The items contained in the personalized collection 
include any combination of shots, takes, scenes, clips, audio 

10 segments, etc. 

Further, the present invention is directed to a system and 
method for providing on demand access to merchandise 
information related to a movie, and for providing on demand 
connectivity to merchants, while the movie is being pre- 

15 sented to a user. This embodiment of the invention operates 
by presenting the movie to the user, receiving from the user 
a merchandise related query pertaining to the movie, and 
determining a scene of the movie that was being presented 
to the user when the user issued the query. Portions of movie 

20 related information relating to merchandise appearing in the 
scene are identified and retrieved. These portions of movie 
related information are then presented to the user. The 
invention enables the user to interact with any merchant who 
provided any merchandise appearing in the scene in 

25 response to a request from the user to do so. 

Further features and advantages of the present invention, 
as well as the structure and operation of various embodi- 
ments of the present invention, are described in detail below 
with reference to the accompanying drawings. In the 

30 drawings, like reference numbers indicate identical or func- 
tionally similar elements. 

The foregoing and other objects, features and advantages 
of the invention will be apparent from the following more 

35 particular description of (a) preferred embodiment(s) of the 
invention, as illustrated in the accompanying drawing(s). 

BRIEF DESCRIPTION OF FIGURES 

The present invention will be described with reference to 
40 the accompanying drawings, wherein: 

FIG. 1 is a block diagram of a preferred embodiment of 
the present invention; 

FIG. 2 is a block diagram of a computer system repre- 
senting a preferred implementation of many components of 
45 the present invention; 

FIGS. 3 and 4 are data flow diagrams depicting the 
manner in which foundation and index information is gen- 
erated according to a preferred embodiment of the present 
50 invention; 

FIGS. 5 and 6 illustrate example index information con- 
tents; 

FIG. 7 illustrates the pedigree of a frame; 
FIG. 8 illustrates an example source table; 
5S FIGS. 9A, 9B, 11, 15, 17A, 18, 19, and 21 are flowcharts 
depicting the operation of the present invention; 

FIGS. 10A-10C, 12A, 12B, 13A, 13B, 14A, 14B, 17B, 
20, 22A, and 22B are example index tables; and 
so FIG. 16 is a block diagram of a manager according to a 
preferred embodiment of the present invention. 

BEST MODE FOR CARRYING OUT THE 
INVENTION 

65 The present invention is directed to a system and method 
for organizing information, and for providing services to 
enable users to efficiently, effectively, and expeditiously 
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access such information. In other words, the present inven- 15. View a movie through a presentation box, to concur- 
tion provides services to enable users to access information rently view the movie and related information of inter- 
in an intelligent, user-driven manner. est. 

For illustrative purposes, the present invention is some- In addition to these queries predominately based on script 
times described herein in the context of information pro- 5 and editing linkages, queries can be made on non-text 
duced by the motion picture industry. That is, the present factors. Metrics extracted from the video, audio, or stills are 
invention is described as a system and method for organizing managed by the database, and are used to form non-text 
movie and supplemental movie information, and for pre- searches against these data. This scheme can be used to 
senting such movie and supplemental movie information to search for non-text aspects such as cut boundaries and 
users. It should be understood, however, that the present 10 movies having similar soundtracks, 
invention is not limited to this embodiment (call the "movie The movie embodiment of the present invention also 
environment" for reference purposes). Instead, the present permits the examination of movie detail beyond that view- 
invention is applicable to any environment where informa- able through a TV or HDTV. Feature films are today 
tion is produced, and where a commercial opportunity exists captured principally on high grade film, affording fidelity far 
with the organization and presentation of such information. 15 beyond that viewable on TV or HDTV. The present inven- 
For example, the present invention is applicable to computer tion permits a viewer to pan and zoom, to examine finer 
programming environment where information about the detail in a movie or image still. 

person who wrote a program, the company that sold the By using the movie embodiment of the present invention, 

program, the place the program was written and tested, etc., film makers are afforded a new level of flexibility in optional 

could be linked to a running program. 20 viewing and associations from within their films. Alternate 

The movie embodiment of the present invention is pref- P lot lines > cuts > or ^nsor levels are much more easily 

erably realized through a combination of a 'true video on constructed, because the film components are manipulated 

demand' infrastructure, database technology, and other spe- independently. Composition is directed through the 

cialized digital servers. By tightly coupling databases to database, easing the construction of alternate presentations 

specialized digital servers for video, audio, still image, text, 25 linkages, 

etc., a novel level of control of movie presentation is STRUCTURE OF THE PRESENT INVENTION 

achieved. Additional information about a movie is stored ™ . • . * , „ , ,. , 

, , . u • x 1 i 1 ne present invention shall now be discussed more een- 

and managed, to permit a viewer to more effectively explore r .. - A „ T „ . , ^ A ,? , 

, j * j • T. u**- c * * i erally with reference to FIG. 1, which illustrates a block 

and understand movies. The combmation of greater control j. J - , A . 

.. . . diagram oi a data processing environment 102 or the present 

and additional information permits the viewer to experience JU . to ™. r . • t 1 i r 

iic.. j * ^ ■ A r ■ invention. 1 his environment 102 includes a plurality or user 

a new level of richness and entertainment from movies, , . c , . . * /o-m\ 

devices 106 representing, tor example, set top boxes (S 1 B) 

Accordingly to the movie embodiment, text, still image, m ^b^on with tc i evisiori monitors, or computers (such 

and audio information is directly related to specific movie M personal ^ ^ dev i C e S 106 each include 

scenes or frames. This level of association, coupled with a 35 a m6chanism (such ^ a key board either integrated into the 

database, permits a greater level of direct access to movie wt devic6S 106 M in a Kmote mntaA miX ) for 

information than previously possible. Searching and cone- receiving control and information messages from human 

lation of supplemental information (also called side operators 

information) permits the examination of different side infer- £ach ^ 10fi ^ a weU ^ transmit 

mation relationships, and permits the following sample . ... ., , , t , 

. ^ r * r . . . , 1 . 40 and receive component to transmit and receive control and 

actions not existing in conventional viewing technologies: . e ,. • i *_ + u a * • • *• 

b & & information signals from other devices via a communication 

1. List the names of the actors, other credited roles, or med i um 124. Preferably, the communication medium 124 
scene technical detail for the current scene (e.g., represents a data communication network, such as a local 
director, producer, musical score, shot location, blue area network or a wide area network, in which case the 
screen, matte, etc.). 45 environment 102 represents a distributed computing envi- 

2. List other movies, TV shows, or other scenes in the ronment. However, the present invention is not limited to 
current movie with one or more properties equal to this connection scheme. For example, some of the devices 
those found above. shown in FIG. 1 may be positioned locally to each other, 

3. list all scenes including a specific product tie-in (i.e., and/or some of the devices in FIG. 1 may be implemented 
product brand name used in movie). 50 using a single computer. In sum, any mechanism or scheme 

4. Save any of these lists for viewing selection. for connecting the devices shown in FIG. 1 is within the 

5. View scene while listening to voice overs of director or scope and spirit of the present invention. 

actor with their comments about the scene. The environment 102 also includes a foundation infor- 

6. View stills with pan and zoom controls, to examine fine mation interface component 108 that controls access to 
detail. 55 foundation inform ation in a foundation information database 

7. View movie at desired censor level. m ( M snown b y line 110 > the foundation information 

8. View script or staging information via picture in inter / ace ^ponent 108 is preferabl) ^ directly connected to 
nicture foundation information database 112.) The term foun- 

n , r ' . c ■., 4 dation information" is used herein to refer to information 

9. View outtakes for the current scene. iL * . . jjt_ ✓ L ^ 

< rt , - . . /. . x .60 that has been produced by one or more parties (each party 

10. View signing (i.e., sign language) via picture in being a persoQ Qr organizaticm) . According to the present 

picture. invention, the foundation information represents digital 

11. Search other stills for like colors, shapes, or textures. information only. The present invention provides foundation 

12. Search the script for a word or phase. information on demand to users via the user devices 106. 

13. Search other scripts for like words or phrases. 65 With respect to the movie embodiment, the foundation 

14. Layout a movie presentation box, with areas for information represents, for example, a movie and supple - 
displaying related information of interest. mental information pertaining to the movie (i.e., additional 
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information generated during the pre-production, client/server model, and makes server requests of the data- 
production, and post-production stages of the movie). The bases 112, 122 and digital servers through the index inter- 
foundation information is described in greater detail below, face component 118 and the foundation information inter- 
Preferably, the foundation interface component 108 rep- face component 108 through an integrated interface. The 
resents a relational database management system (RDBMS). 5 functionality of this integrated interface is represented by the 
The foundation information database 112 preferably repre- presentation and control component 104, although such 
sents a file server. The foundation information database 112 functionality can alternatively be located or distributed 
may be distributed for performance reasons (i.e., to position among other modules. Through these interfaces, the digital 
foundation information closer to the user devices 106 so that video, audio, and still image objects stored on the digital 
communication performance is enhanced), in which case 1Q servers) appear to the application to be managed as data in 
there would be multiple foundation information interface the relational databases 108, 118. 

components 108 and multiple foundation information data- The index information database 122 and the foundation 

bases 112 (for simplicity purposes, only one foundation information database 112 are preferably implemented as 

information interface component 108 and one foundation digital servers for at least two reasons: 

information database 112 are described). 1. To permit the use of specialized hardware and software 

An index interface component 118 controls access to 1 to cost-effectively deliver the digital video, audio, and 

index information in an index information database 122. (As image stills. 

shown by line 120, the index interface component 118 is 2. To permit the distribution of these servers, thereby 
preferably directly connected to the index information data- reducing the distance to the viewer and the associated 
base 122.) This index information represents indices of the latency and transmission costs, 
foundation information in the foundation information data- M noted above > each user device 106 can be a ful1 
base 112. Accordingly, the organization of the foundation function PC, or a more limited function settop box. Depend- 
information is established by the index information. The m § ™ me h "^ ar ? ^software capacity of any particular 
present invention uses the index information to locate and user dev ^ ™ 6 > j ne Ration executing thereon 
r £ , j x . rr-.j may be distributed over the user device 106 and an inter- 
reference particular foundation information. The index „- j- * * *• 1 *• t n j * *• t j 
. _ .. ... j. 1 1 25 mediate station emulating a mil function end station. Inde- 
information is discussed further below. pendent of ^ of the ^ device/ 

Preferably, the index interface component 118 represents intermediate station interacts with both the relational 

a relational database management system (RDBMS). The databases 112, 122 and the digital movie servers 108, 118 

index information database 122 preferably represents a file through the integrated interface (i.e., the presentation and 

server which has special searching capabilities or another 30 con trol component 104) 

relational database management system (RDBMS). The ^ environment 102 also includes a cross merchant 

index information 122 may be distributed for performance gateW ay 116 and a merchant transaction processor 114, 

reasons, in which case there would be multiple mdex inter- whic h are described in U.S. patent application entitled 

face components 118 and multiple mdex information data- System and Method for Providing Merchant Information 

bases 122 (for simplicity purposes, only one index interface 35 md Establishing Links to Merchants While Presenting a 

component 118 and one index information database 122 are Movie", Attorney Docket No. ST9-94-046 (1252.1920000), 

described). referenced above. 

The environment 102 further includes a presentation and The user devices 106, foundation information interface 
control component 104. The functionality of the presenta- component 108, index interface component 118, presenta- 
tion and control component 104 may be distributed, in which 40 tion and control component 104, foundation information 
case there would be multiple presentation and control com- database 112, index information database 122, cross mer- 
ponents 104. For simplicity purposes, only one presentation cna nt gateway 116, and merchant transaction processor 114 
and control component 104 is discussed. The presentation are each preferably implemented using a computer system 
and control component 104 coordinates the actions of the 202 as shown in FIG. 2. (Alternatively, each computer 
user devices 106, the foundation information interface com- 45 system 202 is used to implement a subset of these devices/ 
ponent 108, and the index interface component 118, components.) The computer system 202 includes a proces- 
In particular, the presentation and control component 104 sor 204 (or multiple processors 204) communicating with 
receives information requests from the user devices 106. The other components via a communication bus 206. 
presentation and control component 104 processes these Connected to the bus 206 is a primary memory 208 (such 
information requests by accessing the index information in 50 as random access memory, RAM) which stores control logic 
the index information database 122 (via the index interface 210 (i.e., software) and data 212 (such as a portion of the 
component 118) to identify and locate the particular foun- foundation information and index information, for 
dation information indicated by the user requests. The example), During run-time, the control logic 210 enables the 
presentation and control component 104 then retrieves this processor 204 to perform the functions described herein, 
foundation information from the foundation information 55 Thus, the control logic 210 represents a controller of the 
database 112 (via the foundation information interface com- processor 204. In an alternate embodiment, the functionality 
ponent 108), and presents this foundation information to the of the present invention described herein is achieved using 
users via the user devices 106. In an alternative embodiment, a hardware implemented state machine, 
any or all of the functionality of the presentation and control The computer system 202 also includes secondary 
component 104 is embedded in the index interface compo- eo memory 214, which may include a storage device 216, such 
nent 118 and/or the foundation information interface com- as a removable disk drive (i.e., a floppy drive). A removable 
ponent 108, such that the user devices 106 (in some storage medium 218, such as a computer disk (i.e., a floppy 
circumstances) communicate directly with the index inter- disk) can be inserted in the storage device 216. The control 
face component 118 and/or the foundation information inter- logic 210 is stored in direct computer readable form (i.e., 
face component 108. 65 magnetically or optically) on a storage medium (i.e., mag- 
In an embodiment of the present invention, the user netic storage medium) in the removable storage medium 
devices 106 are each used as the application driver in a 218. 
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Also connected to the bus 206 is a network interface 222, are in the theatrical version of the movie. There may be 

which provides connectivity to the communication medium multiple versions of a movie (for example, the director's 

124 shown in FIG. 1. version, the unrated version, a signed version, etc.). 

Generally, an edit decision list 414 exists for each version. 

Source, Foundation, and Index Information 5 fh e post-production information 412 also includes the 

The present invention also includes a manager 1602 video 416 and audio 418 associated with each version of the 

which is shown in block diagram form in FIG. 16. The movie > and clock information 420 that is embedded in the 

manager 1602 includes one or more capture and digitizer wid ™£ 6 and audi ° 418 - Each frame j a a mo ™ has a 1™ 

components 304 (for simplicity purposes, only one is 5? de - ^f 5 ? ^ c f* s £P rese °< >he clock information 420, 

- ~ • L„ ' „„ ~1 ™*„ mo m The clock information 420 and time codes are discussed in 

described), one or more index generation components 308 10 t , t , . 

/c . i. . . - i 'j -i ,\ greater detail below. 

(for simplicity purposes, only one is described), a person- to _ ... .. . r . ,„ , 

alized presentation manager 1604, and a personalized col- . J** c f l1 d ^on list 414, video information 416, audio 

lection manager 1606. The manager 1602 and its compo- ^formation 418, and clock information 420 represent pn- 

nents 304, 308, 1604, 1606 are preferably implemented mar y J^ 0 ™** 10 ? 1 492 > T hcrcaS ^ [V^oduchon m^onn^ 

using one or more computer-based devices operating in 15 Uon 404 and thc production information 408 represent 

accordance with instructions from computer software (as secondary information 490 (also called movie related 

shown in FIG. 2). Alternatively, the functionality of at least information) Pnmary information 492 is what is normaUy 

portions of the manager 1602 (as described herein) is Presented and available to the public. Its creation is the 

achieved using predominately hardware, such as a hardware P™? 08 * Performing a process such as that shown in FIG 

implemented state machine. 20 4 Secondary information 490 is generated for the purpose of 

^ , A ^ , A . generating the primary information 492, and/or is generated 

The personalized presentation manager 1604 and the j . f. c \ u . . f ' . Afl ~° • 

f. , . 1 ^ jr 6 ,. ... during the creation or the pnmary information 492, and/or is 

personalized collection manager 1606 are discussed below. . . ~. . . . / AM i iL j ^ 

L, 4 * j * . , . to , . it _ . , related to the primary information 492. In other words, the 

The capture and digitizer component 304 and the index • . r r J AM • * j r * L j 

T iad ■ i_ ii i. j i_ j pnmary information 492 is generated from the secondary 

generation component 308 shall now be described. f c *\. Ann ™ j • r *• i ftA ■ * 

& r 25 intormation 490. The secondary information 490 is not 

The capture and digitizer component 304 and the index generally presented to or available to the public. However, 

generation component 308 are used to generate the founda- often the pilblic would be vcry interested in this secondary 

tion information that is ultimately stored in the foundation informatioo 490, and would be willing to pay a fee to have 

information database 112, and the index information that is access to it ^ present invention is directed to providing 

ultimately stored in the index information database 122. The me public ^ access to this secondary information 492 in 

detailed operation of the capture and digitizer component m intelligentj user-friendly manner. Although the concepts 

304 and the index generation component 308 shall now be of primary information 492 and secondary information 490 

described with reference to a data flow diagram shown in Dave been expla i ne d with reference to the movie 

rivJ- Jm embodiment, it should be understood that primary informa- 

The capture and digitizer component 304 receives source 35 tion 492 and secondary information 490 are generated in 

information 302. The source information 302 represents other contexts, and the present invention is equally appli- 

information that has been produced over time by one or cable to these other contexts. 

more parties (each party being a person or organization), and ^ pre . p roduction information 404, production informa- 
nt would be of value to some end users (i.e., those persons ^ m , and post-production information 412 of the source 
working at the user devices 106). This information may be ^ information 302 can be stored in any number of storage 
stored in any number of storage mediums, such as paper, mediums. For example, diagrams of models, storyboards, 
film, video, compact disk, magnetic tape, computer storage contracts with actors and other persons, location 
d ^ cs ' etc - information, and department work orders are typically 

An example of the contents of the source information 302 stored on paper. Scripts are also stored on paper, but may 

in the movie embodiment of the present invention is illus- 45 additionally be stored in electronic form on computer stor- 

trated in FIG. 4. FIG. 4 also illustrates the process by which age disks (i.e., floppy disks). An edit decision list 414 is 

the source information 302 is generated in the movie stored on paper, and may additionally be stored in the 

embodiment. electronic form used by automated offline editing tools. 

As will be appreciated by persons skilled in the relevant Takes are stored on film. Video 416 is stored on film and/or 

art(s), a motion picture is generally produced in three 50 VCR tapes. Audio 418 is stored on tape, compact disks, 

phases: a pre-production phase 402, a production phase 406, computer storage disks, etc. 

and a post-production phase 410. Source information 302 is Referring again to FIG. 3, the capture and digitizer 

generated during each of these phases. Specifically, pre- component 304 converts the source information 302 to 

production information 404 is generated during the pre- foundation information 306. The foundation information 

production phase 402, production information 408 is gen- 55 306 has the same information content as the source infor- 

erated during the production phase 406, and post-production mation 302, but is in digital form. Thus, the foundation 

information 412 is generated during the post-production information 306 is a digital, electronic representation of the 

phase 410. source information 302. 

The pre-production information 404 includes, for Specifically, the capture and digitizer component 304 

example, diagrams of models, storyboards, contracts with eo captures or otherwise extracts the source information 302 

actors and other persons and organizations associated with from the mediums on which they are stored, digitizes the 

the movie, information on locations, versions of scripts, etc. captured/extracted source information 302, and stores such 

The production information 408 includes, for example, digitized information as the foundation information 306. 

takes, continuity sheets, department work orders, etc. Preferably, the operation of the capture and digitizer com- 

The post-production information 412 includes, for 65 ponent 304 is automated to the greatest extent possible, but 

example, one or more edit decision lists 414. An edit may require some human intervention and control depend- 

decision list 414 specifies the scenes, takes, and/or clips that ing on available technology 
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Ihe capture and digitizer component 304 includes, for 
example, scanners, optical character recognition (OCR) 
devices and processes, data compressors and 
decompressors, data extract and capture devices and 
processes, data converters, etc. The structure and operation 
of these devices and processes are well known to persons 
skilled in the relevant art(s) and, thus, shall not be described 
further. 

In some cases, at least some portions of the source 
information 302 are already in digital, electronic form. Such 
portions of the source information 302 are directly stored as 
foundation information 306, as indicated by data flow line 
312. 

Eventually, it is possible that most or even all of the 
source information 302 will be available in digital, elec- 
tronic form (once this occurs, the role of the capture and 
digitizer component 304 will be reduced or even 
eliminated). This is true for at least two reasons. First, the 
advance in computer technology is making it more and more 
attractive for parties to perform their work using a computer. 
As a result, that work is stored in digital, electronic form. 
Second, information generators who intend to utilize the 
present invention will have an incentive to store their work 
in digital, electronic form (the incentive being the ability to 
eliminate the capture and digitizer component 304). 

The index generator component 308 analyzes the foun- 
dation information 306 and produces index information 310. 
The operation of the index generator component 308 is 
automated to the greatest extent possible, but may include 
user intervention and control (such as keying in index 
information). The index information 310 represents indices 
of the foundation information 306. Accordingly, the orga- 
nization of the foundation information 306 is established by 
the index information 310. For example, suppose the foun- 
dation information 306 includes works by a group of 
authors, and articles written about those works and authors. 
Id this example, the index information 310 may have a first 
table listing by author works available in the foundation 
information 306, a second table listing articles available in 
the foundation information 306 about each author, and a 
third table listing articles available in the foundation infor- 
mation 306 about each work. 

With regard to the movie embodiment, and referring to 
FIG. 5, the index information 310 may include, for example, 
video indices 502, audio indices 504, text indices 506, 
parametric indices 508, and image content indices 510. 
Referring now to FIG. 6, the index information 310 in the 
movie embodiment includes general index table(s) 602 that 
pertain to all movies (such as actor biographies), and index 
table(s) 604, 606 specific to each movie. These tables 602, 
604, 606 in FIG. 6 may each include indices as shown in 
FIG. 5. The index information 310 specific to the movie 
embodiment is discussed in greater detail below. 

Video Frame Pedigree 

Referring now to the movie embodiment of the present 
invention, FIG. 7 generally illustrates the making of a 
movie. 

As is well known, during the production phase 406 a 
series of takes 710, 712, 714 are shot on film 720 (typically 
35 mm film in full-frame super-35 format). Film 720 rep- 
resents "negatives" and, thus, shall be called "negative film 
720" for reference purposes. A take is the sequence of 
frames (such as frame 704) from the clap of a clapboard 
(when the director says " actio n") to when the director says 
"cut" or "cut and print." There are many types of takes, such 
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as a master take, a close up (CU), an extreme close up 
(ECU), etc. Each take has one or more shots, such as shot 
706 in take 712. A shot represents those frames in the take 
which are of value, and which may be used in the movie. 

Each frame in the negative film 720 has an unique time 
code which can be tied to the start of the respective take (i.e., 
from the clap of the clapboard). Thus, the frames in the 
negative film 720 can be identified by reference to time 
codes. 

The negative film 720 is developed and portions are 
printed (they are now positive) and are called "dailys." If the 
director ends a take by saying "cut and print," then that take 
is printed. Otherwise, it is not printed. FIG. 7 shows a 
portion of a daily 722. This daily 722 includes the shot 706 
containing the frame 704. Each frame in the dailys 722 
includes a unique time code. These time codes in the dailys 
722 are the same as the time codes in the negative film 720 
as they were copied in the printing process. 

During the post-production phase 410, a theatrical pre- 
sentation 724 of the movie is produced. The theatrical 
presentation 724 includes a sequence of scenes, such as 
scene 708. Each scene includes one or more shots, such as 
shot 706 in scene 708. The theatrical presentation 724 is 
typically generated by splicing together frames from the 
dailys, such as frame 704 from daily 722. This produces a 
work print of the whole movie. Using the workprint as a 
master, the negative film is cut and spliced to match. The cut 
negative is then printed to produce a timed (color corrected) 
positive. All the prints so far have been contact prints so the 
original time codes are preserved. The theatrical negative 
(not shown) is produced optically (to change aspect ratio, 
make room or sound tracks, etc.) and new time codes are 
assigned for the whole movie. Theatrical presentation 724 
prints are made from that negative. Each frame in the 
theatrical presentation 724 includes an unique time code 
copied from the theatrical negative. These time codes are 
measured from the beginning of the theatrical presentation 
724. Thus, the time codes in the theatrical presentation 724 
differ from the time codes in the dailys 722 and the negative 
film 720. However, the process for converting from the time 
codes in the theatrical presentation 724 to the time codes in 
the dailys 722 and negative film 720, and vice versa, is well 
known. 

Typically, versions of the movie other than the theatrical 
presentation 724 are produced. One such version is the VCR 
video version 702. Video runs at a different speed than film. 
Typically, video runs at 30 fps (frames per second) or 25 fps, 
while film runs at 24 fps. To compensate for this difference, 
additional frames are added to the shots in the VCR video 
version 702. For example, a shot 706' in the VCR video 
version 702 includes five frames, whereas the corresponding 
shot 706 in the theatrical presentation 724 includes four 
frames. Each frame in the VCR video version 702 includes 
an unique time code. These time codes are measured from 
the beginning of the VCR video version 702. Since the 
number of frames per shot differ in the VCR video version 
702 and the theatrical presentation 724, the time codes 
between the VCR video version 702 and the theatrical 
presentation 724 also differ. However, the process for con- 
verting from the time codes in the VCR video version 702 
to time codes in the theatrical presentation 724, and to the 
time codes in the dailys 722 and negative film 720, and vice 
versa, is well known. 

Accordingly, it is possible to trace the lineage of any 
frame 704 through the negative film 720, the dailys 722, the 
theatrical presentation 724, the VCR video version 702, and 
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any other version of the movie not shown in FIG. 7, by using erably the present invention utilizes positives where 
the time code of the frame, and also by using the well known possible, rather than negatives and the time codes match), 
time code conversion processes mentioned above. Thus, information identifying the daily 722 is stored in the 

source title column 816. As discussed above in conjunction 
Source Table 5 with FIG ^ the j mages on tne daily 722 were captured, 

FIG. 8 depicts a source table 802 according to a preferred digitized, and stored as foundation information 306 in the 

embodiment of the present invention. A source table 802 foundation information database 112. Preferably, what is 

exists for each version of a movie (i.e., the theatrical stored in the source tide column 816 is information sufficient 

presentation, the VHS video version, the unrated version, the to locate these digitized images of the daily 722 in the 

director's version, etc.), although the source tables 802 for 10 foundation information database 112 (such as a file name or 

different versions may be very similar to each other. In some address). 

cases, different versions may have the same source table Note that the frames in a scene may originate from a 

802. number of sources. Thus, the source title column 816 is 

The source table 802 is similar to an edit decision list, but blank (or not applicable, N/A) for rows corresponding to 

the source table 802 contains information not found in a 15 scenes. Similarly, the source first frame time code column 

conventional edit decision list. Also, the present invention 812 and the source frame count column 814 (described 

uses the source table 802 in ways that differ from the ways below) are also blank. 

that an edit decision list is used. Further, according to the A source first frame time code column 812 stores the time 

present invention, the source table 802 is organized and code of the first frame of the item identified by the source 

stored in electronic form. In fact, the source table 802 is part 20 title column 816. Assume, again, that the row corresponds to 

of the index information that is stored in the index infor- the take 712 in FIG. 7. The source first frame time code 

mation database 122. column 812 stores the time code of the first frame of the take 

The source table 802 specifies the scenes, takes, and/or 712. The time code stored in the source first frame time code 

clips that are in a version of a movie. The source table 802 column 816 is measured from the beginning of the item 

contains sufficient information so that the lineage of each 25 , identified by the source title column 816, i.e., from the 

frame in the movie can be identified and located (an example beginning of the take 712. 

of a frame's lineage or pedigree is shown in FIG. 7). The A source frame count column 814 stores a number that is 

source table 802 represents a linkage mechanism so that equal to the number of frames in the item identified by the 

foundation information 108 related to each frame can be source title column 816. 

identified, located, and retrieved from the foundation infor- 30 a "for which scene" column 818 contains information 

mation database 112. that identifies the scene in which the item corresponding to 

The source table 802 includes a row for each scene, clip, the row is used. For example, if the row corresponds to a 

and take in the movie version. A clip is a sequence of frames take, then the "for which scene'* column 818 contains 

taken without regard to scene, shot or take boundaries. 35 information that identifies the scene in which the take is 

However, it is taken from a single source with a uniform used. This column is blank for rows corresponding to clips, 

time base (such as theatrical presentation 724) so that the . 

start time code and end time codes have unique meanings. User-Driven Operation of the Present Invention 

A type code column 804 in the source table 802 identifies The user-driven operation of the present invention shall 

whether the row is a scene, clip, or take, and identifies the ^ now be described in conjunction with a flowchart depicted 

type of take (master, close up, extreme close up, etc.). ■ in FIG. 9A. This "user-driven" operation differs from the 

A first frame time code column 806 stores the time code operation of the invention as shown in FIG. 3, Specifically, 

of the first frame of the item corresponding to the row. Thus, FIG. 3 illustrates the manner in which the present invention 

if the row corresponds to a scene, then the first frame time generates foundation information 306 and index information 

code column 806 stores the time code of the first frame of 45 310. The user-driven operation depicted in FIG. 9A refers to 

the scene. The time code stored in the first frame time code the manner in which the present invention access, retrieves, 

column 806 is measured from the beginning of the movie and presents foundation information 306 from the found a- 

version to which the source table 802 is associated. The tion information database 112 to users in response to infor- 

present invention preferably' uses SMPTE (Society of mation requests and/or queries from users. 

Motion Picture Television Experts) time codes, but other 50 For illustrative purposes, the user-driven operation of the 

time codes could alternatively be used. present invention shall be described with reference to the 

A last frame time code column 808 stores the time code movie embodiment. It should be understood, however, that 

of the last frame of the item corresponding to the row. Thus, the present invention is not limited to the movie embodi- 

if the row corresponds to a scene, then the last frame time ment. The following description is applicable when the 

code column 808 stores the time code of the last frame of the 55 present invention is used in other applications, 

scene. The time code stored in the last frame time code The flowchart in FIG. 9 A begins with step 950, where 

column 808 is measured from the beginning of the movie control immediately passes to step 952. 

version to which the source table 802 is associated. i n ste p 952, in response to an appropriate request from an 

A frame count column 810 stores a number that is equal user via an user device 106, the presentation and control 

to the number of frames in the item corresponding to the eo component 104 transfers to the user device 106 a list of 

row. Thus, if the row corresponds to a scene, and the scene items which the user may view and/or interact with. Such 

has 79 frames, then the frame count column 810 stores "79." items include, for example, various versions of movies, 

A source title column 816 stores information identifying audio tracks related to movies, video games related to 

the source of the item corresponding to the row. Referring to movies, text from books or magazines related to movies, etc. 

FIG. 7, suppose that the row corresponds to the take 712. 65 Such items also include personalized movie presentations 

The source of this take 712 is the daily 722 (actually, the and personalized movie collections, which are described 

ultimate source of take 712 is negative film 720, but pref- below. 
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The presentation and control component 104 preferably 
performs step 952 by accessing the index information data- 
base 122 (via the index interface component 118) and/or the 
foundation information database 112 (via the foundation 
information interface component 108) for a listing of pos- 5 
sible viewing choices. This list of viewing choices is trans- 
ferred from the presentation and control component 104 to 
the user device 106 via the communication medium 124. 
The user device 106 displays this list of viewing choices in 
a well known manner. 1Q 

In step 954, the presentation and control component 104 
receives from the user (via the communication medium 124 
and the user device 106) information identifying a particular 
item which the user wishes to view. 

In step 956, the presentation and control component 104 
accesses and retrieves the item identified by the user. In one 15 
embodiment, an Item Locator table (which may be imple- 
mented as a collection of tables) is stored in the index 
information database 122. The item Locator table has a row 
for each of the possible viewing choices. Each row stores at 
least (1) the name of the item (or other information to 20 
unambiguously identify the item), and (2) address informa- 
tion identifying the location in the foundation information 
database 112 where the item is stored. 

The presentation and control component 104 performs 
step 956 by accessing the Item Locator table (via the index 
interface component 118) locating the row corresponding to 
the item identified by the user, extracting the address infor- 
mation from this row, and using this address information to 
access and retrieve the item from the foundation information 3Q 
database 112 (via the foundation information interface com- 
ponent 108). The item is transferred to the presentation and 
control component 104 via the communication medium 124. 
For example, if the item is a movie, then the digitized frames 
of the movie are transferred from the foundation information ^ 
database 112 to the presentation and control component 104. 

In step 958, the presentation and control component 104 
begins to present the item to the user by transferring con- 
secutive parts of the item to the user device 106. It is noted 
that the item is transferred to a location specified by the user 
(such as the user's home). Also, the item is preferably 
transferred to this location immediately after the user has 
selected the item. Thus, the present invention implements 
true video on demand. 

For illustrative purposes, the following discussion 45 
assumes that the item is a movie. Thus, during step 958, the 
presentation and control component 104 sends the first 
digitized frame to the user device 106, where the frame is 
displayed. 

In step, 960, the presentation and control component 104 50 
determines whether it has received a query pertaining to the 
movie from the user (via the user device 106 and the 
communication medium 124). The manner in which the user 
sends a query is described below. If a query has been 
received, then the presentation and control component 104 55 
processes the query in step 962, and then proceeds to step 
964. The manner in which queries are processed are 
described below. Step 964 is also processed if no query was 
received. 

In step 964, the presentation and control component 104 eo 
determines if there are any additional frames in the movie 
that have not yet been sent to the user. If additional frames 
exist, then control returns to step 958 where the next frame 
is sent to the user device 106. Otherwise, operation of the 
flowchart of FIG. 9Ais complete, as indicated by step 966. 55 

The above-described operation of the present invention 
can be modified in any number of ways to achieve better 
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performance. For example, according to an alternate 
embodiment, the movie is sent directly to the user device 
106 from the foundation information database 112, rather 
than being sent first to the presentation and control compo- 
nent 104. Also, in this alternate embodiment, the entire 
movie is sent to the user device 106 all M one time, rather 
than being sent frame by frame. 

In another alternate embodiment, personalized movie 
presentations and personalized movie collections are 
retrieved from the foundation information database 112 and 
presented to the user according to index information con- 
tained in personalized edit decision lists. Personalized movie 
presentations and personalized movie collections are 
described further below. 

Sending and Interpreting User Queries 

The user can send a query to the presentation and control 
component 104 at any time while viewing and/or interacting 
with a movie (or some other item). Preferably, the user 
signals a query by pressing the "Pause" button provided by 
the user device 106 (this "Pause" button may be situated on 
the remote control unit, on the set top box, or on the 
computer keyboard, for example). By pressing the "Pause" 
button, the user indicates that he has a question regarding the 
movie, scene, cut, and/or frame that is currently being 
displayed. The question may be, for example, "Who is this 
person?," "What other movies has this director done?/' 
"Why is the street in this street wet when it appears to be a 
clear day?," or "What other movies have script lines similar 
to what was just said?" The questions which users are 
allowed to ask are implementation dependent. Thus, the 
particular questions discussed herein are presented for illus- 
trative purposes only, and are lot limiting. 

Preferably, the user identifies his specific question by 
traversing an user menu hierarchy. User menus are well 
known. Thus, the specifics of the user menu hierarchy used 
to identify user questions shall not be discussed in detail. 

It is noted, however, that the user menu may be driven and 
controlled by the presentation and control component 104 by 
exchanging control and data signals between the presenta- 
tion and control component 104 and the user device 106. 
Alternatively, software control logic and/or hardware con- 
trol logic can be incorporated into the user device 106 such 
that the user device 106 drives and controls the user menu. 

Note that the question "Who is this person?" is somewhat 
ambiguous, in that it could mean "Who is the principle 
actor?," or "Who is the stunt double actor? or "Who is the 
character?" In one embodiment of the invention, this ambi- 
guity is solved by having a very detailed menu structure, so 
that the precise question is identified by the user. However, 
users may find this approach somewhat tedious. 

Accordingly, to an alternate embodiment, the present 
invention makes assumptions as to what the users' precise 
questions are. For example, if the user asks "Who is this 
person?," the present invention may assume that the user is 
really asking "Who is the character in this scene?" The 
assumptions made by the present invention in this alternate 
embodiment are implementation dependent and are based on 
many factors, such as the context in which the question was 
asked, market information, surveys, empirical use, etc. For 
illustrative purposes, this alternate embodiment of the inven- 
tion (where assumptions are made as to the user's precise 
question) is described below. The assumptions that are 
employed in the following description are used only by way 
of example, and may differ depending on the particular 
implementation. 
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Processing User Queries stores the name of the actor, a take title column 1006 stores 

nr> . a^„r„u„~+ iU „ „„„ • the name of the take, a start time code column 1008 stores 

FIG. 9B presents a flowchart depicting the manner in , . * , t n r /• . i • . 

. . . . - /• the time code of the first frame of the take in the current 

which the present invention processes an user query (i.e., the . , ( , . , , 4 iL 

manner in which the present invention performs step 962 in movie and die end tune code column 1010 stores the time 

FIG. 9A). Hie flowchart in FIG. 9B begins with step 902, 5 ^ of last L framc of te ^ e m , the ™ l m ° v f ^ 

where control immediately passes to step 904. co6 * s ™ the start I mc code column 1008 and the end 

, , . , . . A1 . time code column 1010 are measured from the start ot the 

In step 904, the user device 106 determines the time code cwKat moyic Information for thc Actor ^ Xake tablc 1002 

corresponding to the current frame (due time code is called ^ dcriv6d b ^ iQdex tor omat m b refcr . 

the "current frame time code for reference purposes). The pro duction schedule information (that is part of the 

user device 106 then transfers the curren frame Umecode to duct £ n Mom&tioa m y 

the presentation and control component 104 via the com- * T „ ' 1<11V , , ' . 

munication medium 124. (The user device 106 may send the ™ e Character In Scene table 1012 m A cludes a r ° w for " ch 

current frame time code such that the presentation and character who a PP ears m each «-fne. Ascene column 1014 

control component 104 receives it at the same time as „ stores the name of the scene, and a character column 1016 

receiving the user query, i.e., at step 960 in FIG. 9 A.) 15 * to f res the name ° f a character * ho ap P e , ar ! " 

„ * . , „ , Z c .u . t- Informauon for the Character In Scene table 1012 is derived 

The "current frame represents the frame that was being ^ Mex (or Qenl 308 by reference to the 

presented on the user device when the user issued the query cArmt 

(i.e., pressed the "Pause" button). As noted above, clock J* ' . 

information 420 (i.e., time code information) is embedded in „ ™ e Actor ^V 5 Character table 1018 includes, for each 

the video data 416 of the movie (see FIG. 4). Accordingly, c ^ arac [ er m the c ™ movie > a r ™ h a ?!° r wh ° 

in step 904, the user device 106 extracts the current frame ^ ih * character. A character column 1020 stores the name 

time code corresponding to the current frame from the video of the character, an actor column 1022 stores the name of an 

data 416 of the movie being presented to the user. actor who P 1 ^ the character and a code column 1024 

, r j • A r *t_ * * * identifies the manner in which the actor played the character 

The current frame ume code is measured trom the start oi 9 r 4l _ . . , . *u * * j ui +u u j 

iL • / ,* i , r , ^ (i.e.. as the principle actor, as the stunt double, as the body 

the movie (rather than being measured from a daily 722 or ). ' , \ T * *■ c *t> a * m * * ui 

ni **t\ r i \ -n- j double, etc.). Information for the Actor Flays Character table 

negative film 720, for example). Thus, time code conver- * A io-j-jt-«t.-j * * mo u 

. & c ' . L ii 1018 is derived by the index generator component 308 by 

sions are performed in subsequent steps. c . .« ,. 4 r * a 7u a »• 

, . , r + nA reference to the credits of the movie and the production 

In step 906, the presentation and control component 104 schedule 

accesses the index information database 122 (via the index 30 ' . . A a , t 

interface component 118) and extracts from the source table U FI ?' « de P lc ^. a fl 7° hart ^presenting the manner m 

802 associated with the movie being presented (called the ***ch }hc present invention processes a who ,s it? query 

current movie for reference purposes) all rows where the to ° De embodiment. This flowchart begins ^ wim 

current frame time code is greater than or equal to the first ste P U02 > where control ""mediately passes to step 1106. 

frame time code, and the current frame time code is less than 35 In ste P U° 6 > the presentation and control component 104 

or equal to the last frame time code. retrieves from the Actor In Take table 1002 rows that relate 

In step 908, the presentation and control component 104 t0 ^ frame - Ste P "° 6 * PM&med by reference to 

identifies the query type or category. This query type is £e rows were retrieved from the source table 802 m step 

indicated by information contained in the user query Ste P ™*« Performed by retnevmg from the Actor In 

received in step 960 of FIG. 9A (it is assumed that the user 40 Ti * e J?* 2 a ^ r0WS 8 ' ^ ( 

selected a query type by traversing an user menu hierarchy, ?* nnn 1008 ) tf^* 6 s ' art ' ime 00(16 (&°m column 806 

as discussed above). Each implementation of the present m the source^ table 802) in the take row retrieved in step 906 

invention supports a set of query types. For illustrative 11111 •*» havin 8 an end time code (from ^column 1010) equal 

purposes, a selected set of query types is discussed below. to end lime ^ ( from "hunn 808 m the source table 

Other query types will be apparent to persons skilled in the 45 802 > m *« ^ ""f^. m st ^P 9 L 06 " ^ r u ows from 

relevant art(s) the Actor In Take table 1002 identify the actors who appear 

In step 910, the presentation and control component 104 in me current frame, 

processes the user query according to the type of the query . Step 1104 tests to see that at least one actor 1004 was 

as identified in step 908. The manner in which the presen- ^eluded m that take. 0t might have been a long scenery shot 

tation and control component 134 processes a selected set of so &r example ) If no actors are found in the take then the 

query types is discussed below. The processing of other S0O P e 15 *" dened '? mclude a11 ^ charaaers m the entire 

query types will be apparent to persons skilled in the scene and Processing conunues at step 1120. Otherwise, 

relevant art(s) processing contmues with step 1108. 

The flowchart of FIG. 9B is complete after step 910 is In r ste P U08 > the Presentation and control component 104 

performed, as indicated by step 912. 55 transfers to the user device 106 via the communication 

medium 124 the names of the actors stored in the actor 

Query Type: Who Is This? column 1004 of these rows retrieved from the Actor In Take 

This section discusses the "who is this?" query type. table 1002. The user device 106 presents these names to the 

According to an embodiment of the present invention, the user in a well known manner. 

index information database 122 stores at least three index eo In step 1110, the presentation and control component 104 

tables to support the processing of this query type: an Actor retrieves all rows from the Actor Plays Character table 1018 

In Take table 1002, a Character In Scene Table 1012, and an associated with all actors identified in step 1106 (i.e., actors 

Actor Play Character table 1018. These tables 1002, 1012, corresponding to rows retrieved from the Actor In Take 

1018 represent movie specific tables, and are shown in Table 1002). Step 1110 is performed by reference to the actor 

FIGS. 10A-10C 65 column 1022 in the Actor Plays Character table 1018. 

The Actor In Take table 1002 includes at least one row for In step 1112, the presentation and control component 104 

each actor appearing in each take. An actor column 1004 transfers to the user device 106 via the communication 



12/03/2003, EAST Version: 1.4.1 



6,0< 

19 

medium 124 the names of the characters stored in the 
character column 1020 of these rows retrieved from the 
Actor Plays Character table 1018. The user device 106 
presents these names to the user in a well known manner. 

In step 1114, the presentation and control component 104 
retrieves from the Actor Plays Character table 1018 all rows 
with code 1024 value of "principal" related to the characters 
identified in step 1110 (i.e., the characters corresponding to 
the rows retrieved from the Actor Plays Character table 
1018). Step 1114 is performed by retrieving from the Actor 
Plays Character table 1018 all rows corresponding to these 
identified characters. 

In step 1116, the presentation and control component 104 
transfers to the user device 106 via the communication 
medium 124 the names of the actors (column 1022) and their 
roles (column 1024) from these rows retrieved from the 
Actor Plays Character table 1018. The user device 106 
presents this information to the user in a well known manner. 

If, in step 1104, it is determined that no actor may be in 
the current frame, then step 1120 is performed. In step 1120, 
the presentation and control component 104 retrieves all 
rows from the Character In Scene table 1012 associated with 
characters appearing in the current scene. The "current 
scene" is the scene in which the current frame is contained, 
and is identified by reference to information stored in the 
"for which scene" column 818 in the take row previously 
retrieved from the source table 802 (in step 906). Step 1120 
is performed by retrieving from the Character In Scene table 
1012 all rows corresponding to the current scene (by refer- 
ence to the scene column 1014 in the Character In Scene 
table 1012). 

In step 1122, the presentation and control component 104 
transfers to the user device 106 via the communication 
medium 124 the names of the characters stored in the 
character column 1016 of these rows retrieved from the 
Character In Scene table 1012. The user device 106 presents 
these names to the user in a well known manner. After step 
1122 is performed, then control moves to step 1114, 
described above (although, in this case, step 1114 is per- 
formed based on the results of step 1120). 

The operation of the flowchart in FIG. U is complete after 
step 1116 is performed, as indicated by step 1118. 

Query Type: What Other Films Has This Director 
Worked On? 

This section discusses the "what other films has this 
director worked on?" query type. According to an embodi- 
ment of the present invention, the index information data- 
base 122 stores at least two index tables to support the 
processing of this query type: an One Movie Credit table 
1202 (which is movie specific) and an All Movies Credit 
table 1208 (which applies to all movies). These tables 1202, 
1208 are shown in FIGS. 12Aand 12B. 

The One Movie Credit table 1202 includes at least one 
row for each person involved in the creation of the movie 
(for example, a row for each person listed in the credits). 
Each row includes a person column 1204 storing the name 
of the person, and a job column 1206 indicating the person's 
position. If a person had many jobs, then there is one row in 
the One Movie Credit table 1202 for each job. Information 
for the One Movie Credit table 1202 is derived by the index 
generator component 308 by reference to the credits of the 
movie, and also by reference to contracts (pre-production 
information 404). 

The All Movies Credit table 1208 includes one row for 
each job that each person had in each movie represented in 
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the foundation information database 112. A movie title 
column 1212 stores a movie/s name, a person column 1214 
stores a person's name, a job column 1216 stores a job that 
the person had in the movie, and a table identifier row 1210 

5 stores information that identifies where other index tables 
specific to the movie are stored in the index information 
database 122. Information for the All .Movies Credit table 
1208 is derived by the index generator component 308 by 
reference to the credits of the movie, and also by reference 

10 to contracts (pre-production information 404). 

The present invention processes a "what other films has 
this director worked on?" query type as follows. First, the 
presentation and control component 104 accesses the One 
Movie Credit table 1202 for the current movie and retrieves 

15 the row where the job column 1206 is equal to "Director/' 
Second, the presentation and control component 104 
extracts the name of the director from the person column 
1204 of this row. Third, the presentation and control com- 
ponent 104 accesses the All Movies Credit table 1208 and 

20 retrieves all rows where the person column 1214 is equal to 
the director's name. Fourth, the presentation and control 
component 104 extracts the movie title (from the movie title 
column 1212) and the job (from the job column 1216) from 
these rows and sends them to the user device 106 via the 

25 communication medium 124. The user device 106 displays 
this information in a well known manner. 

Query Type: When Did This Happen? 

3Q This section discusses the "when did this happen?" query 
type. According to an embodiment of the present invention, 
the index information database 122 stores at least two index 
tables to support the processing of this query type: a Time In 
Story table 1302 and a Take Details table 1312, both of 
which are movie specific. These tables 1302, 1312 are 
shown in FIGS. 13 A and 13B. 

The Time In Story Table 1302 has a row for each scene 
in the current movie. A scene column 1304 stores the name 
of the scene, a time since story start column 1306 that stores 

4Q a value representing the time (in terms of the story) that has 
elapsed since the current movie started, a time since last 
scene column 1308 that stores a value representing the time 
(in terms of the story) since the last scene ended, and an 
absolute date/time column 1310 that stores a value that 

45 represents the date and time (in terms of the story) of the 
events depicted in the current scene. Information for the 
Time In Story table 1302 is derived by the index generator 
component 308 by reference to time references in the script 
(for example, time indicators such as "that night," "later that 

50 day/* "next morning," etc.). 

The take details table 1312 has a row for each take. A take 
title column 1314 stores the title of the take, a take first 
frame time code column 1316 stores the time code of the 
take measured from the beginning of the current movie, a 

55 location name column 1318 stores the name of the location 
where the take was shot, a date shot (created) column 1320 
and a time shot (created) column 1322 store a date and time 
when the take was shot, and a method column 1324 stores 
information identifying the manner in which the take was 

so shot (i.e., steady-cam, digital morph, etc.). Information for 
the Take Details Table 1312 is derived by the index genera- 
tor component 308 by reference to the production schedule. 

The present invention processes a "when did this hap- 
pen?" query type as follows. First, the presentation and 

65 control component 104 identifies the current scene by ref- 
erence to the "for which scene" column 818 in the take row 
previously retrieved from the source table 802. Second, the 
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presentation and control component 104 retrieves from the 
Time In Story Table 1302 the row where the scene column 
1304 is equal to the name of the current scene. Third, the 
presentation and control component 104 extracts the time 
and date information from columns 1306, 1308, and 1310 of 5 
this row, and forwards this information to the user device 
106 via the communication medium 124. The user device 
106 displays this information in a well known manner. 

The present invention alternatively processes a "when did 
this happen?" query type as follows. First, the presentation 10 
and control component 104 retrieves from the Take Details 
table 1312 the row where the value in the take first frame 
time code column 1316 is equal to the value in the first frame 
time code 806 of the take row previously retrieved from the 
source table 802. Second, the presentation and control 15 
component 104 extracts the date and time information from 
columns 1320 and 1322 of this row, and forwards this 
information to the user device 106 via the communication 
medium 124. The user device 106 displays this information 
in a well known manner. 20 

Other queries can be answered by reference to the Take 
Details Table 1312. For example, a "Where was this take 
shot?" query can be answered by reference to the location 
name column 1318, and a "How was this take shot?" query 
can be answered by reference to the method column 1324. 25 

Query Type: Why Queries 

A "why" query is supported only if a corresponding 
answer is available. An example why question is: "Why is 3Q 
the street in this scene wet when it appears to be a clear 
day?" Atypical answer is: "The director of photography uses 
water to create more interesting images on film by capturing 
the reflections of other parts of the scene in the street. The 
director calls for a wet-down just before the shop and a water 35 
truck drives through the scene and makes it wet." 

The tables and methods for answering these "why" ques- 
tions are shown in FIG. 14A, FIG. 14B and FIG. 15. 
Preferably, the present invention includes a Question/ 
Answer table 1404 and Question Applies to Scene table 4Q 
1424 to support "why" questions. These tables 1404, 1424 
are movie specific, and are stored in the index information 
database 122. Information for the Question/Answer table 
1404 and Question Applies to Scene table 1424 is derived by 
the index generator component 308 by reference to many 45 
sources, such as interviews with persons involved in the 
making of the movie, diaries kept during the making of the 
movie, books describing the making of the movie, etc. 
Preferably, a human operator uses this information to com- 
pose "why" questions and their answers, and to insert these 50 
questions and answers in the Question/Answer table 1404. 
Similarly, a human operator uses this information to deter- 
mine appropriateness of each question to particular scenes or 
sequences of scenes and inserts this information into the 
Question Applies to Scene table 1424. 55 

The Question/Answer table 1404 includes a row for each 
why question that is supported. A Question Number column 
1406 uniquely identifies the question, a question column 
1408 stores the why question, and an answer column 1410 
stores an answer to the question. 60 

The Question Applies to Scene table 1424 includes a row 
for each segment of a movie to which a question applies. A 
Question Number column 1426 identifies the question, a 
First Scene column 1428 identifies the first scene in a 
sequence of scenes for which this question is appropriate, 65 
and a Last Scene column 1430 identifies the last scene in a 
sequence of scenes for which this scene is appropriate. Since 
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questions can apply to several scenes in a movie, many rows 
may have the same Question Number 1426. Many questions 
will apply to any given scene so First Scene 1428 and Last 
Scene 1430 values may also be repeated. 

The present invention processes a why question as is 
shown in FIG. 15. The procedure gets control at 1510 and 
proceeds immediately to the first processing step 1520. First 
the presentation and control component 104 accesses the 
row of source table 802 where the type code 804 is "scene" 
and the current time code of the movie is between the First 
Frame Time Code 806 and the Last Frame Time Code 808 
to retrieve the correct for. which scene 818 value. Second, 
step 1522, the Question Applies to Scene table is accessed 
to retrieve a list of question numbers 1426 where the 
previously retrieved for which scene 818 value is between 
the First Scene 1428 and Last Scene 1430. There is a 
possibility of duplicate question number 1426 values in the 
list and these should be removed. Third, step 1512, the 
questions 1408 from the Question/Answer table 1404 along 
with the questions numbers 1406 are forwarded to the user 
device 106 through the communication medium 124 and are 
subsequently displayed in a well known manner. Upon 
receipt of a response from the user device 106, the question 
1408 and answer 1410 corresponding to the question num- 
ber 1406 selected by the user are retrieved 1514. Finally, 
step 1516, the question 1408 and answer 1410 are forwarded 
to the user device 106 via the communication medium 124. 
The user device 106 displays this information in a well 
known manner. Processing then proceeds directly through 
termination 1518. 

Show Option 

According to an embodiment of the invention, the user 
can request that the script be displayed as the movie is 
presented. The present invention processes this user request 
as follows. First, the time code for the current frame is 
determined. Second, the row corresponding to the current 
scene is retrieved from the source table 802. This is done by 
identifying a scene row in the source table 802 where the 
current frame time code is greater than or equal to the value 
of the first frame time code column 806, and less than or 
equal to the value of the last frame time code column 808. 
Based on the information contained in this scene row, it is 
possible to synchronize the display of the movie with the 
display of the script (the script is retrieved from the foun- 
dation information database 112). 

The operation of the show option shall now be discussed 
in greater detail. The identified row (described above) is the 
first of a sequential set of scene rows which describe the rest 
of the movie. Each has values for frame count 810 and for 
which scene 818. The Script text is accessed using the for 
which scene 818 values obtained from the source table 802 
as index keys. The frame count 810 from each row tells 
exactly how long that scene will be on the screen. The 
scrolling rate of the script text is adjusted so that the script 
rolls by in the same amount of time that the scene takes in 
the movie. Each new segment of the script is accessed to 
have it ready when it has to start appearing on the screen. 
Since the scrolling window has many lines of the script on 
it, the slug line for the following scene will appear before the 
video for the current scene has completed. Well known 
techniques for anticipatory buffering enable smooth scroll- 
ing and synchronization. 

Manager 

As noted above, the present invention includes a manager 
1602 which is shown in block diagram form in FIG. 16. The 
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manager 1602 includes one or more capture and digitizer 1702. In this capacity, the personalized presentation man- 
components 304 (for simplicity purposes, only one is ager 1604 acts as an editor, such that the user can edit the 
described), one or more index generation components 308 rows in the personalized edit decision list 1702. However, 
(for simplicity purposes, only one is described), a person- the personalized presentation manager 1604 functions as 
alized presentation manager 1604, and a personalized col- 5 more than an editor, since it allows a user to view any shot, 
Lection manager 1606. The capture and digitizer component talcCj scene? clip> etc ^ coa tained in any version of the movie, 
304 and me mdex generation component 308 were described or in mc undcr i ymg dai i ys 112 or even the underlying 
above. Tlie personalized presentation manager 1604 and the tfve film ?20 ^ shots> ^ ^ 
personalized collection manager 1606 shall now be dis- located by ^ the associatcd time codcs ^ a rc f er ence to 
cusse * 10 the source table 802 for the movie. Based on information 
Personalized Presentation Manager contained in the source table 802, it is possible to identify the 
The personalized presentation manager 1604 enables a st ° ra S c lo t c ^ on in the fo ™ dat i°° information database 112 
user to create a personalized version of a movie. The user, of an y . shot ' tekc - cll P- etc " ** Massed above, 
in his personal movie version, may include shots not in the Accordingly, the user can view his proposed change to the 
theatrical presentation or the VCR video version, and/or may 15 personalized edit decision list 1702 before actually making 
exclude shots, takes, clips, and/or scenes that are in the cnange. 

theatrical presentation and/or the VCR video version. The ^ operation of the flowchart in FIG. 17A is complete 

user may change the sequence of shots, takes, clips, and/or after sle P 1706 ^ performed, as indicated by step 1708. 

scenes, and/or may change the length of shots, takes, clips, FIG. 18 presents a flowchart representing the manner in 

and/or scenes. Other modification options available to users 20 which the presentation and control component 104 presents 

will be apparent to persons skilled in the relevant art based a. personalized movie to a user. This flowchart begins with 

on the discussion contained herein. step 1802, where control immediately passes to step 1804. 

The user interacts with the personalized presentation In step 1804, the presentation and control component 104 
manager 1604 to create a personalized movie version. In one selects and retrieves the first entry (row) from the person- 
embodiment, the personalized presentation manager 1604 alized edit decision list 1702 for the personalized movie the 
forms part of the presentation and control component 104, user wants to view. 

such that interaction with the user is achieved through the In step 1806, the presentation and control component 104 

transfer of control and data messages between the presen- uses the information contained in this row to retrieve from 

tation and control component 104 and the user device 106. 3Q the foundation information database 112 the digitized frames 

In an alternate embodiment, the personalized presentation corresponding to the clip represented by this row (that was 

manager 1604 is part of the presentation and control com- retrieved in step 1804). In particular, the presentation and 

ponent 104, but becomes resident (at least in part) at the user control component 104 uses information from the source 

device 106 pursuant to a user request to create or edit a tide column 1712, the source first frame time code column 

personalized movie version. This alternate embodiment is 35 1708, and the source frame count column 1710 from this 

advantageous in that it reduces traffic on the communication row (retrieved from the personalized edit table in step 1804) 

medium 124. Other ways for implementing the personalized to obtain these digitized frames. 

presentation manager 1604 will be apparent to persons In step 1808, the presentation and control component 104 

skilled in the relevant art(s). presents these digitized frames to the user. In particular, the 

FIG. 17A presents a flowchart depicting the operation of ^ presentation and control component 104 transfers these 

the personalized presentation manager 1604 when working digitized frames to the user device 106 via the communica- 

with an user to create or modify a personalized movie tion medium 124, and the user device 106 displays the 

version. This flowchart begins with step 1702, where control frames in a weU known manner. Note that, while these 

immediately passes to step 1704. frames are being presented to the user, the user can issue a 

In step 1704, the personalized presentation manager 1604 45 query, as described above. Any such query issued by the user 

creates a copy of the edit decision list corresponding to the is processed in the same manner as described above, 

movie which the user wishes to personalize. This copy is i n ste p 1810, the presentation and control component 104 

called the personalized edit decision list for reference pur- determines whether there are any more entries in the per- 

poses. An example of a personalized edit decision list 1702 sonalized edit decision list 1702 left to process. If there are 

is shown in FIG. 1702. 50 more en tries to process, then in step 1814 the next entry is 

According to a preferred embodiment of the present selected, and control returns to step 1806. Otherwise, opera- 
invention, a personalized movie version is made up of a tion of the flowchart in FIG. 18 is complete as indicated by 
sequence of clips. Accordingly, the personalized edit deci- step 1812. 
sion list 1702 includes a row for each clip. The columns in 

the personalized edit decision list 1702 have the same 55 Personalized Collection Manager 

function as those of the same name in the source table 802 The personalized collection manager 1606 enables a user 

(FIG. 8). Thus, once editing of the personalized edit decision to create a personalized collection of items for later presen- 

list 1702 has been completed, and the personalized movie is tation. These items may be scenes and/or clips from a movie 

being viewed, it is possible to process user queries in the or from multiple movies, tracks from a soundtrack or 

same manner as discussed above. 60 multiple soundtracks, favorite "coming attraction" clips, 

It may be necessary to perform a translation from the etc., or any combination of these. Thus, the personalized 

movie's edit decision list, which includes scene and take collection manager 1606 enables a user to create collections 

entries, to the personalized edit decision list 1702, which or groupings of items of interest. 

contains just clip entries. The mechanics of this translation The user interacts with the personalized collection man- 
will be apparent to persons skilled in the relevant art(s). 65 ager 1606 to create a personalized collection. In one 
In step 1706, the personalized presentation manager 1604 embodiment, the personalized collection manager 1606 
enables the user to modify the personalized edit decision list forms part of the presentation and control component 104, 
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such that interaction with the user is achieved through the 
transfer of control and data messages between the presen- 
tation and control component 104 and the user device 106. 
In an alternate embodiment, the personalized collection 
manager 1606 is part of the presentation and control com- 
ponent 104, but becomes resident (at least in part) at the user 
device 106 pursuant to a user request to create or edit a 
personalized collection. This alternate embodiment is 
advantageous in that it reduces traffic on the communication 



ner similar to that discussed above in conjunction with FIG. 
18. While viewing a personalized collection, an user can 
issue a query as discussed above. The present invention 
processes such a query in the manner discussed above. 

While various embodiments of the present invention have 
been described above, it should be understood that they have 
been presented by way of example only, and not limitation. 
Thus, the breadth and scope of the present invention should 
not be limited by any of the above-described exemplary 
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medium 124. Other ways for implementing the personalized 10 embodiments, but should be defined only in accordance with 

collection manager 1606 will be apparent to persons skilled the following claims and their equivalents, 

in the relevant art(s). While the invention has been particularly shown and 

FIG. 19 presents a flowchart depicting the operation of the described with reference to (a) preferred embodiments) 

personalized collection manager 1606 when working with thereof, it will be understood by those skilled in the art that 

an user to create or modify a personalized collection. This 15 (various changes) (the foregoing and other changes) in form 

flowchart begins with step 1902, where control immediately and details ma y be made therein without departing from the 

passes to step 1904. s P irit and scope of ^ e invention. 

In step 1904, the personalized collection manager 1606 ^ h A at fe t . , . ■ u 

creates a new personalized collection table, or opens an * meth ° d °/ eDablm S a USer . t0 °P™1* with a person- 

- 20 ahzed version of a movie, comprising the steps of: 

(1) enabling said user to modify an edit decision list 
associated with said movie, said edit decision list 
having entries for at least some of at least one of scenes, 
clips, and takes in said movie, said modified edit 
decision list storing at least information sufficient to 
locate movie related information pertinent to genera- 
tion of said movie; and 

(2) presenting to said user said personalized version of 
said movie in response to a request from said user; 

wherein step (1) comprises the steps of: 

receiving from said user information identifying a 

particular portion of said movie; 
retrieving information corresponding to said particular 

portion of said movie; and 
enabling said user to add any part of said retrieved 
information corresponding to said particular portion 
of said movie to said personalized version of said 
movie. 

2. The method of claim 1, wherein step (2) comprises the 
steps of: 

(a) retrieving an entry from said modified edit decision 
list; 

(b) retrieving one or more digitized frames specified by 
said retrieved entry; and 

(c) presenting said digitized frames to said user. 

3. The method of claim 1, further comprising the step of: 

(3) receiving from said user a query pertaining to said 
personalized version of said movie. 

4. The method of claim 3, wherein said query is one of the 
following query types: 

(a) a "who is this?" query; 

(b) a "what other films has this director worked on?" 
query; 

(c) a "when did this happen?" query; and 

(d) a "why?" query. 

5. The method of claim 4, wherein processing of a "who 
is this?" query involves accessing one or more of an Actor 
in Take table, a Character In Scene table, and an Actor Play 

60 Character table, and wherein processing of a "what other 
films has this director worked on?" query involves accessing 
one or more of an One Movie Credit table and an All Movies 
Credit table, and wherein processing of a "when did this 
happen?" query involves accessing one or more of a Time In 
The presentation and control component 104 presents a 65 Story table and a Take Details table, 
personalized collection by traversing each entry in the 6. A system of enabling a user to operate with a person- 

corresponding personalized collection table 2002 in a man- alized version of a movie, comprising: 



existing personalized collection table for editing (which 
action is taken depends on the user's request). FIG. 20 
illustrates an example personalized collection table 2002, 
which has an entry (row) for each item in the collection. The 
source first frame time code column 2004, source last frame 
time code column 2006, and source title column 2008 are 
similar to the identically named columns in the source table 
802 (FIG. 8). These columns 2004, 2006, 2008 contain 
information necessary to locate the underlying foundation 
information in the foundation information database 112. The 
personalized collection table 2002 shown in FIG. 20 is 
somewhat specialized for takes, cuts, scenes, and/or clips 
from a movie. Minor modifications to the personalized 
collection table 2002 may be necessary to accommodate 
other items, such as audio tracks (for example; for audio 
tracks, the source first frame time code column 2004 and 
source last frame time code column 2006 are modified to 
store time codes or other indicia to identify the beginning 
and end of an audio track). The nature and implementation 
of such modifications will be apparent to persons skilled in 
the relevant art(s). 

In step 1906, the personalized collection manager 1606 
enables the user to modify the personalized collection table 
2002. In this capacity, the personalized collection manager 
606 acts as an editor, such that the user can edit the rows in 
the personalized collection table 2002. However, the per- 
sonalized collection manager 606 functions as more than an 
editor, since it allows a user to preview any shot, take, scene, 
clip, audio segment, etc., contained in any version of the 
movie, the underlying dailys 722, the underlying negative 
film 720, or other foundation information stored in the 
foundation information database 112. These shots, takes, 
scenes, clips, audio segments, etc., are located by using the 
associated time codes or other identifying indicia as a 
reference to the source table 802 for the movie, or to other 
index tables stored in the index information database 122. 
Based on this index information, it is possible to identify the 
storage location in the foundation information database 112 
of any shot, take, scene, clip, audio segment, etc., as 
discussed above. Accordingly, the user can view his pro- 
posed change to the personalized collection table 2002 
before actually making the change. 

The operation of the flowchart in FIG. 19 is complete after 
step 1906 is performed, as indicated by step 1908. 
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edit decision list modifying means for enabling said user 
to modify an edit decision list associated with said 
movie, said edit decision list having entries for at least 
some of at least one of scenes, clips, and takes in said 
movie, said modified edit decision list storing at least 
information sufficient to locate movie related informa- 
tion pertinent to generation of said movie; and 

presenting means for presenting to said user said person- 
alized version of said movie in response to a request 
from said user; 

wherein said edit decision list modifying means com- 
prises: 

means for receiving from said user information iden- 
tifying a particular portion of said movie; 

means for retrieving information corresponding to said 
particular portion of said movie; and 

means for enabling said user to add any part of said 
retrieved information corresponding to said particu- 
lar portion of said movie to said personalized version 
of said movie. 

7. The system of claim 6, wherein said presenting means 
comprises: , 

means for retrieving an entry from said modified edit 
decision list; 

means for retrieving one or more digitized frames speci- 
fied by said retrieved entry; and 
means for presenting said digitized frames to said user. 

8. The system of claim 6, further comprising: 

means for receiving from said user a query pertaining to 
said personalized version of said movie. 

9. The system of claim 8, wherein said query is one of the 
following query types: 

(a) a "who is this?" query; 

(b) a "what other films has this director worked on?" 
query; 

(c) a "when did this happen?" query; and 

(d) a "why?" query. 

10. The system of claim 9, wherein processing of a "who 
is this?" query involves accessing one or more of an Actor 
in Take table, a Character In Scene table, and an Actor Play 
Character table, and wherein processing of a "what other 
films has this director worked on?" query involves accessing 
one or more of an One Movie Credit table and an All Movies 
Credit table, and wherein processing of a "when did this 
happen?" query involves accessing one or more of a Time In 
Story table and a Take Details table. 

11. A computer program product having control logic 
stored therein, said control logic, when executed by a 
computer, enabling the computer to enable a user to operate 
with a personalized version of a movie, said control logic 
comprising: 

edit decision list modifying means for enabling the com- 
puter to enable said user to modify an edit decision list 
associated with said movie, said edit decision list 
having entries for at least some of at least one of scenes, 
clips, and takes in said movie, said modified edit 
decision list storing at least information sufficient to 
locate movie related information pertinent to genera- 
tion of said movie; and 

presenting means for enabling the computer to present to 
said user said personalized version of said movie in 
response to a request from said user; 

wherein said edit decision list modifying means com- 
prises: 

means for enabling the computer to receive from said 
user information identifying a particular portion of 
said movie; 
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means for enabling the computer to retrieve informa- 
tion corresponding to said particular portion of said 
movie; and 

means for enabling the computer to enable said user to 
add any part of said retrieved information corre- 
sponding to said particular portion of said movie to 
said personalized version of said movie. 

12. The computer program product of claim 6, wherein 
said presenting means comprises: 

means for enabling the computer to retrieve an entry from 

said modified edit decision list; 
means for enabling the computer to retrieve one or more 

digitized frames specified by said retrieved entry; and 
means for enabling the computer to present said digitized 

frames to said user. 

13. The computer program product of claim 6, wherein 
said control logic further comprises: 

means for enabling the computer to receive from said user 
a query pertaining to said personalized version of said 
movie. 

14. The computer program product of claim 8, wherein 
said query is one of the following query types: 

(a) a "who is this?" query; 

(b) a "what other films has this director worked on?" 
query; 

(c) a "when did this happen?" query; and 

(d) a "why?" query. 

15. The computer program product of claim 9, wherein 
processing of a "who is this?" query involves accessing one 
or more of an Actor in Take table, a Character In Scene table, 
and an Actor Play Character table, and wherein processing 
of a "what other films has this director worked on?" query 
involves accessing one or more of an One Movie Credit 
table and an All Movies Credit table, and wherein processing 
of a "when did this happen?" query involves accessing one 
or more of a Time In Story table and a Take Details table. 

16. A method for enabling a user to operate with a 
personalized collection comprising one or more shots, takes, 
scenes, clips, video segments, audio segments, and multi- 
media segments, said method comprising the steps of: 

enabling said user to modify a personalized collection 

table having entries for items in said personalized 

collection; and 
presenting to said user at least a part of said personalized 

collection in response to a request from said user, 

comprising the steps of: 

retrieving an entry from said personalized collection 
table; 

retrieving information specified by said retrieved entry; 
and 

presenting said retrieved information to said user. 

17. A system for enabling a user to operate with a 
personalized collection comprising one or more shots, takes, 
scenes, clips, video segments, audio segments, and multi- 
media segments, comprising: 

modifying means for enabling said user to modify a 

personalized collection table having entries for items in 

said personalized collection; and 
presenting means for presenting to said user at least a part 

of said personalized collection in response to a request 

from said user, comprising: 

means for retrieving an entry from said personalized 

collection table; 
means for retrieving information specified by said 

retrieved entry; and 
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means for presenting said retrieved information to said 
user. 

18. A computer program product having control logic 
stored therein, said control logic when executed by a com- 
puter causing the computer to enable a user to operate with 
a personalized collection comprising one or more shots, 
takes, scenes, clips, video segments, audio segments, and 
multimedia segments, said control logic comprising: 

modifying means for causing the computer to enable said 
user to modify a personalized collection table having 
entries for items in said personalized collection; and 
presenting means for causing the computer to present to 
said user at least a part of said personalized collection 
in response to a request from said user, comprising: 
means for causing the computer to retrieve an entry 

from said personalized collection table; 
means for causing the computer to retrieve information 

specified by said retrieved entry; and 
means for causing the computer to present said 

retrieved information to said user. 

19. A system for presenting movies, comprising: 
a communication medium; 

one or more foundation information databases coupled to 

said communication medium; 
one or more index information databases coupled to said 

communication medium; and 
one or more presentation and control components coupled 

to said communication medium. 

20. The system of claim 19, further comprising: 
one or more cross merchant gateways coupled to said 

communication medium; , and 
one or more merchant transaction processors coupled to 
said communication medium. 

21. A method of presenting movie related objects, com- 
prising the steps of: 

retrieving a next portion of a selected movie related object 
by reference to a foundation information database; 

presenting said next portion of said selected movie related 40 
object; 

receiving a query pertaining to said selected movie related 
object; and 

processing said received query by reference to said foun- 
dation information database. 

22. The method of claim 21, wherein said received query 
is one of the following query types: 

(a) a "who is this?" query; 

(b) a "what other films has this director worked on?" 
query; 

(c) a "when did this happen?" query; and 

(d) a "why?" query. 

23. The method of claim 21, wherein said selected movie 
related object is one of a version of a movie, a movie audio 
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track, a video game related to a movie, text related to a 
movie, or a personalized collection. 

24. A system of presenting movie related objects, com- 
prising: 

means for retrieving a next portion of a selected movie 
related object by reference to a foundation information 
database; . 

means for presenting said next portion of said selected 

movie related object; 
means for receiving a query pertaining to said selected 

movie related object; and 
means for processing said received query by reference to 

said foundation information database. 

25. The system of claim 24, wherein said received query 
is one of the following query types: 

(a) a "who is this?" query; 

(b) a "what other films has this director worked on?" 
query; 

(c) a "when did this happen?" query; and 

(d) a "why?" query. 

26. The system of claim 24, wherein said selected movie 
related object is one of a version of a movie, a movie audio 
track, a video game related to a movie, text related to a 
movie, or a personalized collection. 

27. A computer program product having control logic 
stored therein, said control logic when executed by a com- 
puter causing the computer to present movie related objects, 
said control logic comprising: 

means for enabling the computer to retrieve a next portion 

of a selected movie related object by reference to a 

foundation information database; 
means for enabling the computer to present said next 

portion of said selected movie related object; 
means for enabling the computer to receive a query 

pertaining to said selected movie related object; and 
means for enabling the computer to process said received 

query by reference to said foundation information 

database. 

28. The computer program product of claim 27, wherein 
said received query is one of the following query types: 

(a) a "who is this?" query; 

(b) a "what other films has this director worked on?" 
query; 

(c) a "when did this happen?" query; and 

(d) a "why?" query. 

29. The computer program product of claim 27, wherein 
said selected movie related object is one of a version of a 
movie, a movie audio track, a video game related to a movie, 
text related to a movie, or a personalized collection. 
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